Section: New Results

Visual Neuroscience

Shifting stimulus for faster receptive fields estimation of ensembles of neurons

Participants : Bruno Cessac, Matthias Hennig [University of Edinburg, UK] , Gerrit Hilgen [Institute of Neuroscience, Medical School, Newcastle University, Newcastle, UK] , Pierre Kornprobst, Daniela Pamplona, Sahar Pirmoradian [University of Edinburg, UK] , Evelyne Sernagor [Institute of Neuroscience, Medical School, Newcastle University, Newcastle UK] .

The Spike Triggered Average (STA) is a classical technique to find a discrete approximation of the Receptive Fields (RFs) of sensory neurons  [63] , a required analysis in most experimental studies. One important parameter of the STA is the spatial resolution of the estimation, corresponding to the size of the blocks of the checkerboard stimulus images. In general, it is experimentally fixed to reach a compromise: If too small, neuronal responses might be too weak thus leading to RF with low Signal-to-Noise-Ratio; on the contrary, if too large, small RF will be lost, or not described with enough details, because of the coarse approximation. Other solutions were proposed consisting in starting from a small block size and updating it following the neuron response in a closed-loop to increase its response  [70] , [78] , [77] . However, these solutions were designed for single cells and cannot be applied to simultaneous recordings of ensembles of neurons (since each RF has its own size and preferred stimulus).

To solve this problem, we introduced a modified checkerboard stimulus where blocks are shifted randomly in space at fixed time steps. This idea is inspired from super-resolution techniques developed in image processing [84] . The main interest is that the block size can be large, enabling strong responses, while the resolution can be finer since it depends on the shift minimum size. In [52] , we show that the STA remains an unbiased RF estimator and, using simulated spike trains from an ensemble of Linear Nonlinear Poisson cascade neurons, it was predicted that this approach improves RF estimation over the neuron ensemble, in terms of resolution and convergence. In [53] , we test these predictions experimentally on the RFs estimation of 8460 ganglion cells from two mouse retinas, using recordings performed with a large scale high-density multielectrode array. We compare RFs obtained using (i) the classical checkerboard stimulus with block size of 160$\mu$m and (ii) our checkerboard stimulus with block size of 160$\mu$m and arbitrary shifts of 40$\mu$m in $x-$ and $y-$directions. Results show how spatial resolution can be improved and that our approach allows to recover 51% of the mapped RFs at a resolution of 40$\mu$m, while in the classical case, 41% of the RFs could be found at a resolution of only 160$\mu$m. Thus, our approach improves not only the quality of the RF estimation but also the amount of successfully mapped RFs in neural ensembles.

This work was presented in [52] , [53] and it is being used in current experimental protocols by E. Sernagor (Newcastle University), partner of the EC IP project FP7-ICT-2011-9 no. 600847 (RENVISION).

Using neural mechanisms underlying motion analysis for optical flow estimation

Participants : Manuela Chessa [University of Genoa, DIBRIS, Italy] , Pierre Kornprobst, Guillaume S. Masson [Institut de Neurosciences de la Timone, Team InVibe] , Kartheek Medathati, Fabio Solari [University of Genoa, DIBRIS, Italy] .

We explore how motion information, also called optical flow, is estimated from natural moving sequences. Owing to application potential, optical flow estimation has been studied extensively by computer vision. On the other hand the neural mechanisms underlying motion analysis in the visual cortex have been extensively studied almost with little interaction with computer vision community resulting in few mathematical models. Even though there was some early interaction among the two communities for example, methods by Heeger et.al, Sejnowski et. al, comparatively little work has been done in terms of examining or extending the mathematical models proposed in biology in terms of their engineering efficacy on modern optical flow estimation datasets.

Pursuing this idea, in [26] , we proposed a neural model inspired from the ones presented in  [87] , [86] which are popular models of primate velocity encoding. We started from a classical V1-MT feedforward architecture. We modeled V1 cells by motion energy (based on spatio-temporal filtering), and MT pattern cells (by pooling V1 cell responses). The efficacy of this architecture and its inherent limitations in the case of real videos were not known. To answer this question, we proposed a velocity space sampling of MT neurones (using a decoding scheme to obtain the local velocity from their activity) coupled with a multi-scale approach. After this, we explored the performance of our model on the Middlebury dataset. To the best of our knowledge, this is the only neural model in this dataset. The results were promising and suggested several possible improvements, in particular to better deal with discontinuities. An extension was proposed in [40] .

We also focused on the decoding the motion energies which is of natural interest for developing biologically inspired computer vision algorithms for dense optical flow estimation. In [37] , we addressed this problem by evaluating four strategies for motion decoding: intersection of constraints, maximum likelihood, linear regression on MT responses and neural network based regression using multi scale-features. We characterized the performances and the current limitations of the different strategies, in terms of recovering dense flow estimation using Middlebury benchmark dataset widely used in computer vision, and we highlight key aspects for future developments.

This work was partially funded by the EC IP project FP7-ICT-388 2011-8 no. 318723 (MatheMACS).

Bio-Inspired Computer Vision: Towards a Synergistic Approach of Artificial and Biological Vision

Participants : Pierre Kornprobst, Guillaume S. Masson [Institut de Neurosciences de la Timone, Team InVibe] , Kartheek Medathati, Heiko Neumann [Ulm University, Germany] .

Studies in biological vision have always been a great source of inspiration for design of computer vision algorithms. In the past, several successful methods were designed with varying degrees of correspondence with biological vision studies, ranging from purely functional inspiration to methods that utilise models that were primarily developed for explaining biological observations. Even though it seems well recognised that computational models of biological vision can help in design of computer vision algorithms, it is a non-trivial exercise for a computer vision researcher to mine relevant information from biological vision literature as very few studies in biology are organised at a task level.

In [42] , we aim to bridge this gap by providing a computer vision task centric presentation of models primarily originating in biological vision studies. Not only we revisit some of the main features of biological vision and discuss the foundations of existing computational studies modelling biological vision, but also consider three classical computer vision tasks from a biological perspective: image sensing, segmentation and optical flow. Using this task-centric approach, we discuss well-known biological functional principles and compare them with approaches taken by computer vision. Based on this comparative analysis of computer and biological vision, we present some recent models in biological vision and highlight a few models that we think are promising for future investigations in computer vision. To this extent, this paper provides new insights and a starting point for investigators interested in the design of biology-based computer vision algorithms and pave a way for much needed interaction between the two communities leading to the development of synergistic models of artificial and biological vision.

[42] is under review. This work was partially funded by the EC IP project FP7-ICT-388 2011-8 no. 318723 (MatheMACS).